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ABSTRACT 

Canadian universities are struggling to address seemingly contra- 
dictoiy challenges pertaining to undergraduate education: high demand 
and underfunding. A number of instruments, including the National 
Survey of Student Engagement (National Survey of Student Engage- 
ment, n.d.), have led to greater priority being placed on the undergradu- 
ate experience. Yet, strategies to ensure student satisfaction with their 
education, through initiatives such as small classes and personal contact 
with faculty, seem at odds with the large classes necessitated by fiscal 
imperatives. We carried out a systematic investigation of the impact of 
one problem-based learning course on first year students’ experiences. 
We also investigated the persistence of skills and attitudes learned in 
this single exposure to problem-based learning. The results of our inves- 
tigation show that this course had very positive effects on the immediate 
and persistent behaviours of students. Our research provides empirical 
evidence of the effectiveness of problem-based learning and leads us to 
suggest how a problem-based approach might help universities enhance 
the quality of education and the undergraduate experience. 

RESUME 

Les universites canadiennes sont aux prises avec deux defis 
apparemment contradictoires en matiere d’enseignement au premier 
cycle : une demande elevee et un financement insuffisant. Plusieurs 
instruments, dont un sondage sur I’engagement etudiant (National 
Survey of Student Engagement), ont accru I’importance accordee a 
I’experience etudiante au premier cycle. Neanmoins, les strategies 
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mises en oeuvre pour accroitre la satisfaction, comme les classes de 
petite taille et les relations individualisees avec les enseignants, vont a 
rencontre des mesures imposees par les contraintes financieres. Nous 
avons realise une enquete sur fimpact sur rexperience etudiante de 
premiere annte d’un cours base sur I’apprentissage par problemes. Nous 
avons aussi etudie la persistance des competences et attitudes acquises 
suite a cette unique experience de I’apprentissage par problemes. Les 
resultats de notre enquete demontrent que ce cours a eu des effets tres 
positifs sur les pratiques immediates et a long terme des etudiantes et 
etudiants. Notre recherche offre des preuves empiriques de refficacite 
de I’apprentissage par problemes. Selon nous, cette approche pourrait 
aider les universites a amdiorer la qualite de leur enseignement et 
rexperience de la population etudiante.. 


INTRODUCTION 

Universities are currently facing the dual challenges of enhancing the qual- 
ity of teaching and learning, particularly at the undergraduate level, while at the 
same time they struggle to mitigate the negative effects of financial constraints, 
enrolment increases, and larger classes. Against this context of competing and 
seemingly irreconcilable priorities, it is difficult to engage in a meaningful dis- 
cussion of curricular and pedagogical innovation without fear that the very 
foundations of postsecondary education are being eroded. Our research invites 
both university administrators and faculty members to step outside their re- 
spective comfort zones to consider one pedagogical method that may address 
both the imperatives of fiscal restraint and enhanced learning. We argue that 
the widespread implementation of problem-based learning across the under- 
graduate curriculum would address both demands, although it would require 
a profound cultural shift in approaches to higher education. For departments 
and faculty members, our research challenges conventional views of teaching 
and learning and discipline-based education. Fundamental notions of coverage, 
demonstration of content-based expertise, and familiar modes of course deliv- 
ery are all questioned. For university administrators, our research suggests the 
need to reassess such structures as course weightings, timetabling, and the types 
of infrastructure provided for teaching and learning. Yet, it is our view that the 
outcomes of problem-based learning, as supported by our research findings, if 
applied widely across the curriculum, would not only overcome resistance, but 
would also address many areas of lingering dissatisfaction in the current orga- 
nization and culture of universities. 

The pedagogical literature is replete with subjective accounts and assertions 
about the benefits or otherwise of active learning strategies and, in particular, 
problem-based learning (Barrows, 1996). There is however a dearth of empirical 
evidence, particularly outside the context of medically-based curricula, about 
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the impacts of problem-based learning. The present paper provides empirical 
data to demonstrate the advantages of problem-based learning within a single 
undergraduate course, compared with other small group experiences, and with 
traditional methods of first-year course delivery. It also provides some insight 
into the persistence of learning techniques and the confidence gained in a prob- 
lem-based course, beyond the initial encounter with problem-based learning, 
throughout the remainder of an undergraduate degree. 

In a major conference on the impacts of problem-based learning in 2000, 
Samford University (Birmingham, Alabama) reported that this approach en- 
riches undergraduate learning, can be conducted in conventionally funded and 
organized undergraduate programs, is better applied across a whole curricu- 
lum than in single courses, requires faculty training, and leaves students more 
enthusiastic about learning. It also suggested that problem-based learning is 
more difficult to apply to undergraduate students outside medical curricula 
(Chapman, Keller & Fournier, 2002). Many of these claims, however, were based 
on subjective observation (Blumberg & Eckenfels, 1988). Our study offers a 
systematic analysis of students’ perceptions of the immediate and longer-term 
effects of learning in a single, interdisciplinary problem-based course offered 
in first-year university. 

All of the students were enrolled in their second semester and, in addition 
to the problem-based course, were enrolled in an array of other traditionally 
taught courses and were carrying a full course load. The specific objectives of 
the research were to explore the students’ perceptions of the value of the prob- 
lem-based course in terms of developing processing and reasoning skills and in 
meeting their expectations. We also explored the transferability and persistence 
of learning skills developed in the problem-based course for the remainder of 
the students’ various programs of study. 

The opportunity to conduct this research was presented when the University 
of Guelph implemented a first-year seminar program. Students who were regis- 
tered in the first year of various baccalaureate degree programs in the arts, sci- 
ences, combined arts and science, and commerce were afforded the opportunity 
to register for small group classes which were designed by senior faculty with 
the express goal to foster learning across disciplines. We designed a course that 
bridged our areas of expertise - Murray is a medieval historian and specialist in 
the history of sexuality and Summerlee is a biomedical scientist with experience 
developing and delivering curricula using problem-based learning methods. In 
this course we sought to explore the cultural and scientific approaches to under- 
standing sex, gender, and sexuality from antiquity to the present. Each problem 
presented to the students was designed to address a coordinated series of process 
and reasoning skills, and to empower the students to learn behavioural traits im- 
portant for continuous learning, as well as to accumulate a body of knowledge 
and develop an understanding of the content. Enrolment in the course was lim- 
ited to 1 6 students who were split into two groups of eight students. All classes 
were held with the eight students and two facilitators (Murray and Summerlee) 
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present. How students perceived the learning process was determined through 
pre- and post-semester surveys, which were compared with the responses of two 
other sets of students: those who had also actively selected a small group experi- 
ence in a different first-year seminar course, and those first-year students who 
were registered in a traditionally taught course. 

We have chosen to report the research in a manner that reflects both the 
way a scientist might report experimental research and the way a humanist 
might analyze and contextualize research. Consequently, we present both quan- 
titative and qualitative data and conclusions. 

PROBLEM-BASED LEARNING 

Our course was designed according to a specific mode of problem-based 
learning: closed-loop reiterative problem-based learning (Barrows, 1986; 
Schmidt, 1983). This is quite distinct from the variations of problem-based 
learning upon which other studies, in particular those reporting negative or 
mixed results, have been based. Closed-loop reiterative problem-based learning 
demands that both faculty and students invest completely in the pedagogy (as 
described in detail below). As a result, both students and the course facilitators 
must trust the methodology and avoid the temptation to pedagogical hybridiza- 
tion; that is, they must adhere completely to a problem-based approach. Our 
adherence to a 100% closed-loop reiterative problem-based learning format sets 
this course apart from others discussed in the literature. For example, at Sam- 
ford University, despite a campus-wide commitment and significant external 
funding to support the conversion of courses and indeed whole programs to 
problem-based learning, of the 132 courses involved, no course was completely 
based on problem -based learning; all were hybrids to a greater or lesser degree 
(Chapman, Keller & Fournier, 2002). Faculty continued to use lectures to set 
the context, to supply information and background material that students were 
assumed to lack, or to synthesize the “answers” to problems with conventional 
content. 

Many courses at Stamford appear to have implemented what might better 
be termed “problem-assisted learning”; they used problems as a means to pro- 
vide small group and/or team experience, or to ensure active student participa- 
tion (Chapman, Keller & Fournier, 2002). In other studies, authors report the 
use of lectures to introduce foundational theories or materials, with problems 
providing case studies for the application of concepts (Furmanski, Kane, Gupta 
& Pruitt, 2006). These hybrid courses also report mixed results. Students re- 
sisted the aspects of the course that required self-directed learning, and feared 
they were missing key content that was delivered by lecturers in conventional 
courses (Chapman, Keller & Fournier, 2002). In a so-called problem-based en- 
gineering course at the University of California, Berkeley, traditional testing of 
standard discipline content was used with unhappy results. Although students 
had developed intellectual self-confidence, they did not perform as highly on 
examination instruments as students from conventionally-taught course sec- 
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tions (Furmanski, Kane, Gupta & Pruitt, 2006). Problem-based learning does 
not lend itself to conventional forms of assessment - nor does hybridization 
necessarily enhance the pedagogy or student learning. Rather, the inclusion of 
some aspects of conventional instruction and evaluation undermines the real 
potential of problem-based learning to develop students’ higher order research, 
analytical and critical skills, and their capacity for integrating knowledge. 

In closed-loop reiterative problem-based learning, students are presented 
with a problem which requires they explore issues that arise, and they research 
and learn in the context of that problem. The ultimate goal is not to solve the 
problem, but rather to use the problem as a starting point to reach a level of un- 
derstanding that integrates research and encourages students to assimilate and 
integrate knowledge. The design of the problem creates additional conditions 
for students to learn how to communicate, criticize, and behave in academic 
situations (Barrows, 1985). The behavioural aspects of this learning process 
are equally as important as the content. Over the length of a course, students 
receive a series of cases or problems that have been constructed to focus the 
students on specific learning issues. Each case has been developed around a 
particular theme, concept or idea that will drive the students to develop a series 
of learning issues around the themes. These learning issues create a web of in- 
formation and define the learning outcomes for the whole course. 

Students, then, are presented with a problem which they discuss, and they 
identify issues they do not understand. These are the learning issues that they 
will subsequently research. The students assume the responsibility to research 
issues that are not understood, to find the required information, to present it to 
the group to discuss, and to use their lack of background as the foundation for 
their learning. 

The value of this method of problem-based learning is that it slows down the 
problem-solving process and articulates its various stages so that all participants 
understand the process and their role in the process, and focuses understanding 
of the students at the sophisticated levels identified in Bloom’s taxonomy of edu- 
cational objectives (Bloom, 1956). When students are presented with a problem, 
they are asked to identify three things: (1) what they know; (2) what they do 
not know; (3) what the best way is to find the information they need. Thus, the 
problem is presented in stages. Students receive a page of information and work 
through the issues that are generated on this page before they proceed to the 
next page. The problem comprises a series of pages and the facilitators serve as 
gatekeepers, managing the speed of the process, and not permitting the group to 
move to a new page until the current page is fully analyzed. As the group reads 
and discusses the information, they will accumulate a long list of learning issues, 
and may become bewildered by the information and daunted by the prospect of 
researching so many learning issues. However, the list of issues can usually be 
simplified and focussed on the basic issues that underlie the problem at hand. 
Students then divide the learning issues amongst themselves and agree to pres- 
ent their research at the next meeting, ideally two to three days hence. 


92 CJHE / RCES Volume 37, No. 3, 2007 


In the second meeting, each student presents a short discussion of his/her 
research. This includes presenting interesting books and articles, pictures, arte- 
facts, or hand-outs that have been prepared for the group. Over time, the quality 
of the presentations, and the degree of critical thinking applied to the resources, 
will develop so that the students become more rigorous in their appraisal of the 
information provided. The information is then integrated into the problem, and 
the process of reading through additional pages resumes. If necessary, the group 
will again pause, distribute additional learning issues, and pursue and present 
further research. 

Every meeting of the group ends with a period of group processing. This 
is an absolutely critical component of the learning process, yet it is also one of 
the aspects that is most frequently absent from various experiments in applying 
problem-based learning techniques. This method of learning requires constant 
feedback and evaluation. At the end of every session, each member of the group 
is asked to provide oral feedback to every other member of the group. Each 
member is also asked to discuss his or her individual performance and contri- 
butions to the group. The feedback must be specific, explicit, and focussed on 
behaviours and actions that helped or hindered the group while working on the 
problem and the learning issues that emerged from it. The role of the facilita- 
tor is to ensure that this aspect of the process is completed at every session 
and to model honest, respectful, and focussed evaluation. The intent of group 
processing is to create a safe space in which students can genuinely appreciate 
how they are seen by others and how their behaviours contribute to the group’s 
learning process. Initially, there will be discomfort and even resistance to group 
processing; however, it is an essential component of problem-based learning. It 
is also critical that the facilitator(s) provide and receive feedback from members 
of the group in the same way as the students. 

Another fundamental principle of closed-loop reiterative problem-based 
learning is ensuring that academic assessment and grading are carried out in the 
same way as the pedagogical approach; for example, it is not effective to teach 
process but then examine content, or to teach using a problem-based approach 
but then examine students using multiple choice tests. In our course students 
were evaluated in two ways. The first evaluation was derived from the students’ 
written self-reflections and assessments of their peers and of the facilitators 
both halfway through the semester and again at the end of the course. The mid- 
point evaluation was used to provide feedback about the process and to help set 
the relative grading expectations, for the final assessment, the grades given by 
the student, his or her peers, and the facilitators were all averaged. This com- 
prised 70% of the final grade. This part of the evaluation consisted of questions 
related primarily to process in identilying and researching learning issues, and 
sharing those issues with the group. In short, the first part of the evaluation of 
student learning focused on process rather than content. 

The second part of the evaluation was based on individual written reports. 
Students were provided with a series of articles from books, newspapers or the 
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Internet and asked to analyze the issues that arose and their wider implications 
in light of the course themes. In this assessment, the facilitators evaluated pro- 
cess, content, and writing ability. Students were encouraged to submit drafts 
or outlines of their arguments in advance to either or both facilitators, who 
provided written feedback and suggestions. The written report comprised 30% 
of the final grade. 


METHODOLOGY 

There were three objectives of the research: (1) to explore the value of the 
problem-based experience compared with the experience of other pedagogical 
formats; (2) to investigate the degree to which problem-based learning pro- 
vides, or does not provide, an advantage in improving students’ processing 
and reasoning skills; and (3) to determine the transferability and persistence of 
learned skills. Two experimental approaches were taken to analyze the impact 
of the course in problem-based learning. 

The first approach was designed to explore the immediate impact of prob- 
lem-based learning on the skills and experiences of first-year students and to 
understand whether their experiences met their expectations. Students were 
asked to complete a survey of their perceptions before they started the course 
and immediately upon completing the course. The questions in the survey fo- 
cussed on three categories: (1) process skills and experiences; (2) knowledge 
and reasoning skills; and (3) expectations of the university experience. The 
questions appear in Tables 1 to 3. Students were asked to rank their experi- 
ences on a scale of 1 to 10 where 5 was considered average. The responses of 
the three groups of students were compared: (a) students who completed our 
problem-based first-year seminar course; (b) students who completed a first- 
year seminar course at Guelph (also small-group based) that was not taught in a 
problem-based manner; and (c) first-year students who were in regular courses 
in the Baccalaureate of Arts and Science program at Guelph. The results of the 
surveys were collated and compared. Significant differences in perception were 
determined using ANOVA and comparing students’ ‘f test scores. 

In the second approach, the students’ perception of the persistence of learn- 
ing habits developed during the first-year seminar program was explored. Stu- 
dents in their third year completed a survey to explore how they perceived 
their experiences in the first-year problem-based class as having affected their 
subsequent approach to, and success at, university. The responses of students 
who had completed the problem-based course were compared with those of 
third-year students who had completed a first-year seminar that was not taught 
in a problem-based manner. The questions posed are shown in Table 4. In this 
survey, students were asked to rate their perceptions of the extent to which 
their experiences in the first-year problem-based course had affected their sub- 
sequent learning abilities and experiences. They were asked to rate according 
to a five-point scale: not at all; some; moderate; significantly; and very signifi- 
cantly. In addition to the survey responses, the students were invited to provide 
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qualitative feedback about their first-year seminar and its impact on their sub- 
sequent educational experiences. 


RESULTS 

The goal of the first approach - the pre- and post-survey of first-year 
students - was to learn about their expectations of university teaching and 
learning and to determine the immediate impact of small group learning on stu- 
dents’ perceptions of processing and reasoning skills. The responses of the three 
groups are shown in Table 1 (perceptions of the impact on processing skills), 
Table 2 (perceptions of the impact on reasoning skills), and Table 3 (perceptions 
of the extent to which the experiences at university met expectations). 

There were 15 student respondents in each group (100% response rate). 
Overall, the responses of the students in the problem-based seminar group show 
a marked and significant improvement in scores compared with the other first- 
year seminar group and the group in conventional courses. These changes be- 
tween pre- and post-survey responses were present in almost every aspect of 


Table 1 : Comparison of perception of processing skills before and after 
semester 



PBL 

group 

Seminar 

group 

Lecture 

group 

How much experience do you have with group work 

* 

0 

0 

How much experience do you have with problem- 
solving 

* 

* 

0 

How much experience do you have with giving 
feedback to people 

* 

0 

0 

How much experience do you have with independent 
research 

* 

0 

0 

How comfortable are you using the library 

* 

0 

0 

How comfortable are you using the internet for 
research 

0 

0 

0 

How comfortable are you assessing the quality of 
research data 

* 

0 

0 

How much contact did you have with faculty last 
semester 

** 

* 

0 

How much experience do you have making class 
presentations 

0 

0 

0 


n = 15 in each group 

* Significant {p <0.05) improvement compared with pre-test 
** Significant [p <0.01) improvement compared with pre-test 
0 No significant change observed 
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Table 2 : Comparison of perception of reasoning skills before and after 
semester 



PBL group 

Seminar 

group 

Lecture 

group 

Your ability to generate ideas 

* 

0 

0 

Your ability to identify problems effectively 

* 

0 

0 

Your ability to engage in constructive debate 

n 

0 

0 

Your ability to participate in discussions 

* 

0 

0 

Your ability to participate in group work 

(*) 

0 

0 

Your experience of group work 

* 

0 

0 

Your abilities to resolve conflict 

n 

0 

0 

Your ability to write coherently 

* 

0 

# 

Your ability to deliver constructive criticism 

n 

0 

0 

Your ability to receive and respond to constructive 
criticism 

* 

# 

# 


n = 15 in each group 

Significant [p <0.05) improvement compared with pre-test 
(*) Trend to improvement compared with pre-test 
0 No significant change observed 

# Significant [p <0.05) reduction in performance compared with pre-test 


Table 3: Comparisons of the expectations of experiences before and after 
semester 



PBL 

group 

Seminar 

group 

Lecture 

group 

In class experience 

# 

* 

0 

Experiences in residence 

0 

0 

0 

Individual/personal contact with faculty 

** 

* 

0 

Amount of active learning rather than passive pre- 
sentation 



# 

Receipt of prompt feedback from faculty on assign- 
ments/evaluations 

** 

0 

0 

Respect for diversify of talent 


0 

# 

Respect for diversify of ways of learning 


0 

## 

Sense that faculty have high expectations of your 
performance 

** 

# 

0 

Evaluation procedures and fairness of the examina- 
tions 


0 

0 


n = 15 in each group 

* Significant (p <0.05) improvement compared with pre-test 
** Significant [p <0.01) improvement compared with pre-test 
0 No significant change observed 

# Significant [p <0.05) reduction in performance compared with pre-test 
## Significant [p <0.01) reduction in performance compared with pre-test 
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students’ perceptions of process and reasoning skills, and in their expectations 
of their university experiences. In particular, the problem-based learning group 
indicated a significant [p < 0.01) change in their direct personal contact with 
faculty, the amount of active versus passive learning, their receipt of direct 
feedback, and the sense that there were high expectations placed on them by 
faculty. In contrast, there were relatively few areas in which students in the 
non-problem-based learning first-year seminar course reported significantly 
improved perceptions. Students who did not participate in the first-year semi- 
nars scored themselves significantly worse in their ability to write coherently 
and to receive and respond to constructive criticism. Their surveys also showed 
that over the course of the semester they perceived a significant decrease in 
their ability to respect diversity of learning. Their responses also indicated that 
there was significantly less active learning in their program compared with 
their expectations. 

For the second part of the investigation, the transferability and persistence 
of skills developed during first-year seminars were explored during the third 
year of the students’ undergraduate programs at the University of Guelph. Two 
groups of 1 5 students were surveyed. An overview of the questions is shown in 
Table 4, and a summary of the results of the survey appear in Figure 1. In gener- 
al, when compared with the group who participated in a small group seminar in 
the first-year but did not participate in the problem-based course, students who 
participated in the problem-based course reported a greater sense of awareness 
that process and reasoning skills learned in the course were transferable to 
learning experiences in other courses, and that these skills persisted throughout 
their university careers. The problem-based students noted that process skills 
such as the ability to work in groups, to be effective problem-solvers, and to 
be comfortable interacting with faculty were transferable and persistent attri- 
butes. They also noted that their abilities to engage constructively in debate, to 


Table 4: Survey questions posed to senior students 


Process Skills and experiences 

Group work 

Problem solving 

Giving feedback to peers 

Comfort with independent research 

Using the library 

Using the internet for research 

Comfort with contact with faculty 

Making class presentations 


Knowledge and reasoning skills 

Ability to identify problems and issues 

Ability to engage constructively in debate 

Ability to participate effective in group work 

Ability to deal with and resolve conflict 

Ability to critique information 

Ability to write coherently 

Ability to deliver constructive criticism to peers 

Ability to receive and respond to criticism 


For each question, students were asked to determine the extent to which the process, knowledge and 
reasoning skills, and experiences had been affected by the problem-based or non-problem-based 
experiences in their first-year seminar. They were asked to respond on a five-point scale: not at all, 
some, moderately, significantly, and very significantly. 
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70 
60 
50 
40 
30 
20 
10 
0 

Not Some Moderate Significant Very significant 

I':-:-:-:- Non problem-based group 
Problem-based group 

Students were asked to rank the extent to which their experiences in more senior courses had been 
affected by their experiences in their first-year seminars in the following categories: group work; 
problem-solving abilities; providing feedback to peers; comfort with independent research; using the 
library; using the internet for research; comfort with contacting faculty; and making class presenta- 
tions. In general, students in the non-problem-based group (dotted histogram) indicated that effects 
of the first-year experience were moderate to significant on later performance. In contrast, students 
in the problem-based group (solid black) reported that their subsequent experiences and skills were 
significantly or very significantly affected by their first-year experiences. 

Process Skills and Experiences 



Problem-based group 

Students were asked to rank the extent to which their abilities to accumulate knowledge and demon- 
strate reasoning skills in more senior courses had been affected by experiences in the first-year semi- 
nar classes. They were asked to grade the impact in the following areas: ability to identify problems, 
engage constructively in debate, participate in group work, deal with and resolve conflict, critique 
information, identify appropriate resources that provide understanding of issues, write coherently, 
and deliver, receive, and respond to constructive criticism. In general, students in the non-prob- 
lem-based group (dotted histogram) reported a moderate impact of their first-year experiences on 
subsequent learning experiences. In particular, they observed a complete absence of influence of the 
first-year seminar on their ability to deliver, receive, and respond to constructive criticism. In con- 
trast, students from the problem-based seminar group (solid black histogram) reported significant 
to very significant impact on their subsequent abilities to accumulate knowledge and demonstrate 
effective reasoning skills. 

Knowledge and Reasoning Skills 

Figure 1: Comparisons of the Transferability and Persistence of Processing Skills 
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critique information, and to receive and respond to criticism were profoundly 
influenced by the problem-based course. Students in the non-problem-based 
group reported that their small-group course in their first year of study had a 
moderate to positive impact (using the five-point scale in the survey) on their 
abilities and skills later in their degree work (Figure 1), with the exception of 
their ability to deliver, receive, and respond to constructive criticism, which 
they reported was not at all affected by the first-year experience. 

When students were given the opportunity to express opinions freely about 
the ways in which the small group experience had influenced the rest of their 
university career and learning experiences, there was a significant difference in 
the responses of the two groups. Twelve of the 1 5 students in the problem-based 
group provided comments that indicated the experience was profound and sig- 
nificantly influential: “I developed a passion for learning” and “I developed 
life-long skills about how to learn” were simple reflections on the experience. 
Many of their musings were more reflective. For example, one student wrote, 
“The most important skill that I learned was to believe that no problem, no is- 
sue was beyond my comprehension.” This same student continued, “I arrived at 
University to do business because I did not believe I could learn science and was 
not interested in history and the arts. This course made me realize that learning 
is a passion and every subject, every aspect of learning about anything is not 
only exciting but fun.” 

The students who took the problem-based course also noted that the course 
provided them with transferable skills. One student wrote, “The skills that I took 
. . . . have been at the heart of my success at University.” Another stated, “This 
course has helped me in my overall studies . . . I learned . . . how to organize 
my thoughts and arguments into coherent sections.” Students identified specific 
skills that they learned and carried into other courses. For example, one person 
remarked, “I learned how to think for myself, how to be independent, how to 
manage my time, and to have confidence in my abilities.” Another commented, 
“I am more comfortable asking questions; I have a greater ability to take in 
information that is being presented and then researching issues afterwards in 
an effective way.” More than one student identified that he/she had learned 
to use the library effectively. Finally, one commented that the experience had 
provided her with “a subtle awareness of the difference between information 
and knowledge.” 

In contrast, only three students from the non-problem-based seminar group 
provided commentary about the transferability of skills that they had learned in 
the seminar class. Their common theme was that they looked forward to more 
“small group classes” which they found more demanding and more interesting 
than large, traditional lecture courses. “The seminar course was the course in 
the semester that provided the most interest [for me], I wish there was more 
opportunity to engage in small group learning and to have more time to study 
material in greater depth.” The students in the traditional course provided com- 
ments that underscored their disappointment with large class size and lack of 
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challenging material to study. For example, one student wrote, “All my first 
year courses were a disappointment. The first part of the course was spent 
making sure that we were all at the same level. For me, this was repetitious 
and boring. After that we started work on new material but never spent time 
on anything in depth. There was an obsession with covering a large amount of 
material and no time to think.” 


DISCUSSION 

The results of our research demonstrate that providing students with an op- 
portunity to be immersed fully in a closed-loop reiterative problem-based learn- 
ing experience profoundly affects their perceptions of their ability to research 
and analyze information, and meets or exceeds their expectations for learning 
in a university setting. Moreover, the skills learned in this single course are 
considered by the students to be transferable and persistent for the remainder 
of their university experience. Also, students believe the problem-based course 
to have a significant effect on their motivation, approach to learning, and or- 
ganizational skills. These students perception of their abilities is significantly 
different compared with that of the students who participated in other first-year 
seminar classes that were not taught in a problem-based manner, and with stu- 
dents who were taught in more conventional, didactic first-year classes. There 
are some disturbing changes noted for students in the more traditional classes 
that are consistent with findings in other surveys of the first-year university 
experience overall. These students identified challenges in writing coherently 
and in appreciating the importance of diversity and tolerance of diversity in the 
classroom (see Tables 2 & 3). 

Our findings are significant for universities currently facing a series of 
interrelated challenges: in particular, the imperatives from government and the 
public alike that universities be accountable for delivering demonstrable value- 
added education that prepares students to succeed in a globalized economy, 
despite a generation of underfunding that places Canadian universities among 
the poorest funded in North America (Rae, 2005). The implementation of Key 
Performance Indicators and benchmarking comes at a time when a university 
education is more highly prized than ever and enrolments have reached historic 
levels. In particular, the various public reports that rank universities, such as 
Macleans’ and the Globe and Mail’s Report Card, add considerable urgency to 
internal discussions about the quality of education and the most effective use 
of resources to achieve quality education. 

Increasingly, Canadian universities are participating in the National Survey 
of Student Engagement (NSSE) in order to assess how students are responding 
to their educational experience. The Ontario government has made participa- 
tion in NSSE mandatory as part of the Reaching Higher plan for postsecondaiy 
education. How individual universities fare within their comparison group will 
affect future investment and program development. There are five benchmarks 
of Effective Educational Practice that underlie the NSSE survey. These are (1) 
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level of academic challenge; (2) active and collaborative learning; (3) student- 
faculty interaction; (4) enriching educational experience; and (5) supportive 
campus environment. For the NSSE survey, first-year and fourth-year students 
are surveyed. This provides a sense of the quality of student experience over the 
course of an undergraduate education. 

When the results of our research on students’ perception of their educa- 
tional experience are compared with the NSSE benchmarks, it is evident that the 
problem-based learning format addresses all five of these areas. Moreover, the 
students in the problem-based learning first-year seminar scored significantly 
higher than either those who were in a small seminar or a traditional course, 
which implies that problem-based learning might address many of the issues 
raised in the NSSE survey. The problem-based learning format provides edu- 
cational opportunities that exceed the simple desire for smaller classes, which 
is often proffered as a panacea for the ills that currently beset universities. 
With problem-based learning techniques, students develop transferable skills 
that persist throughout the course of their program of studies, and they de- 
velop qualities of self-directed learning that endure despite the return to large 
lecture format and content-focussed courses. Moreover, the results reported by 
first-year students in traditional courses are cause for concern, given they re- 
port lower expectations, negative experiences, and an overall decrease in their 
learning abilities over the course of the first year of university studies. 

Recently, at the request of the Council of Ministers of Education of Canada, 
the Ontario Council of Academic Vice Presidents developed a set of Guidelines 
for University Undergraduate Degree Level Expectations, aimed at providing a 
standard against which new and existing undergraduate programs could be mea- 
sured (Ontario Council of Academic Vice Presidents, 2005). The various expecta- 
tions for a baccalaureate degree, regardless of discipline, were set out under a 
series of rubrics: (1) depth and breadth of knowledge; (2) knowledge of method- 
ologies; (3) application of knowledge; (4) communication skills; (5) awareness of 
the limits of knowledge; and (6) autonomy and professional capacity. This frame- 
work is largely based on Bloom’s taxonomy (Bloom, 1956). Again, students who 
participated in the problem-based learning seminar at the University of Guelph 
indicated increased competence in a number of key areas - for example, the 
ability to gather, review, evaluate, and interpret information; the ability to take 
initiative and to undertake a critical evaluation of arguments; and, the ability to 
propose solutions and frame appropriate questions. The students demonstrated a 
superior knowledge of methodologies, application of that knowledge, communi- 
cation skills, an awareness of the limits of knowledge, and an understanding of 
autonomy and responsibility for learning. They were able to use the library effec- 
tively and to critique sources of information in ways that are normally expected 
of more senior students. We contend that problem-based learning techniques 
enhance students’ abilities to succeed in their degree programs and to meet the 
expectations of an undergraduate degree better than traditional lecture-based 
courses, and significantly better than a small class experience alone. 
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There are a number of misconceptions around problem-based learning that 
are used as evidence that this mode of course deliveiy cannot be implemented 
outside well-resourced areas such as medicine, which has low faculty/ student 
ratios and focuses on so-called “real world” applications of learning (Aronow- 
itsch & Craaford, 1995; Bradbeer, 1996; Mennin & Martinez -Burrola, 1986; 
Smith, 1985). In particular, with the tremendous growth in university enrolment, 
coupled with chronic underfunding, there is a widespread perception that low 
enrolment courses are a luxury the contemporary multiversity cannot afford, 
especially at the lower levels of instruction. If one accepts the premise that one 
single problem-based learning course can provide in-depth education in which 
the students are more effectively employed in carrying out their own research 
rather than sitting in class - as evidence would suggest from the research in this 
paper - it might be possible to develop a model for education that is sustain- 
able and financially possible even with the current level of funding. It might be 
necessary to use graduate or senior undergraduate student facilitators to achieve 
this but such models are worth investigating (Duch, Allen & White, n.d.). 

Canadian universities have tended to use a pyramid model to establish class 
size. In first year, classes are very large and then become progressively smaller 
as students proceed through a program of study. By the fourth year, the desire 
to provide a capstone experience for majors leads most disciplines to provide 
small-group learning opportunities. It has been suggested that problem-based 
learning is only successful at the fourth year, and that it is appropriate for ma- 
terial requiring higher order thinking as outlined in Bloom’s Taxonomy (Fur- 
manski, Kane, Gupta & Pruitt, 2006). Our research, and the experience of our 
students, on the contrary, suggests that the fourth year may be too late and that 
problem-based learning should be available at the beginning of students’ uni- 
versity education. The greatest educational value, based on the transferability 
and persistence of skills and self-directed learning strategies, will accrue from 
problem-based learning in first year. Students can hone their skills in a small 
problem-based learning context and then implement them to enhance their 
learning at senior levels, including in lecture-based courses. Again, this may be 
a way to conceive of introducing and using problem-based learning to greatest 
effect and could be financially achievable. 

Problem-based learning is a dynamic process. The same problem will not gen- 
erate identical responses from different groups. Some groups may move quickly 
through one case and need to move more slowly through another. Consequently, 
because of the unique and dynamic character of each group, the time devoted to 
a specific problem cannot be set arbitrarily. This necessary process has been criti- 
cized as dragging on and inhibiting the instructor’s ability to cover a set amount 
of material (Chapman, Keller & Fournier, 2002). Flowever, arguably the most 
important learning outcomes, especially at the first-year level, are process and 
behavioural rather than content and the accumulation of fact-based information. 
Each group needs to have the flexibility to dictate the number of sessions needed, 
according to the background, experience and personalities of the members. 
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Furmanski, Kane, Gupta and Pruitt (2006) suggested that students are not 
aware of their own abilities and limitations in self and peer assessment pro- 
cesses. This is not the experience in our courses and does not accord with the 
work of Barrows (1996) and others. Others have suggested that problem-based 
learning leads to grade inflation (Chapman, Keller & Fournier, 2002). Our ex- 
perience again indicates that this is not the case, providing there is a careful 
discussion of expectations and if, by consistently engaging in group processing, 
students learn how to assess themselves and others. Each time we have collated 
our grades with those that students have assigned to their peers and themselves, 
there has been considerable consistency. Students develop a sense of rigour and 
standards of performance against which they measure themselves and their 
peers in a fair and responsible fashion. Grade inflation is not a temptation 
in a context which values intellectual honesty, hard work, and the quality of 
thought. One of the limitations of our study is that we did not attempt to follow 
the grade performance of students in the three groups reported in the current 
study. It would have been useful to compare the entering and exiting grades of 
students in the different groups. This work is currently underway. 

Universal small-group experience in the first year has been considered too 
expensive for under-resourced institutions. This view is based on traditional no- 
tions of course weighting which count a seminar as but one of the 10 semester- 
long courses that comprise the average first-year student’s course load. Based 
on our experience and feedback from our students, this adherence to traditional 
notions of what counts as one course significantly undervalues problem-based 
learning seminars. Students report devoting more time and energy to this one 
course than to their other four courses combined. Given the learning outcomes 
and the transferability and persistence of skills that result from a problem-based 
learning course, arguably one course such as this can provide a learning experi- 
ence equivalent to multiple large courses based on traditional modes of delivery 
and passive, rote learning; this comment was made frequently by students in 
their overall assessments of the course. By reducing the number of courses 
which students take, resources would be freed to permit universal first-year, 
problem-based learning seminars. 

The implementation of problem-based learning in the first year of univer- 
sity does not so much require increased resources, as a dynamic and radical re- 
thinking of the purpose and goals of the first year. Given the well-demonstrated 
limitations of lectures, including the short-term retention of fact-based infor- 
mation, it is more a question of policy and pedagogy rather than of finances. 
Academic disciplines and departments need to recognize the inefficiency and 
limitations of conventional information transfer in lectures and instead begin 
to appreciate how the development of process and behavioural skills will lead 
to long-term student success. 

There are a number of key factors that are absolutely necessary for prob- 
lem-based learning to succeed. As our research results have shown, not every 
small group experience will deliver the same learning outcomes as problem- 
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based learning that is not partial or hybridized. Moreover, many of the mixed or 
negative results reported in the literature are direetly attributable to modifica- 
tions to the methodology or the complete absence of a critical component, par- 
ticularly the group processing. To be successful, problem-based learning must 
be implemented in all its aspects. 

One critical factor in the success of problem-based learning is the size of 
the group. In our experience the optimum size of a group is eight students plus 
facilitator(s), although we have also had success with groups of seven or nine 
students. Some initiatives report groups as small as four to six (Chapman, Keller 
& Fournier, 2002; Dion, 1996; Rhem, 1998), while others do not discuss the size 
of the groups, suggesting a lack of attention to this aspect of the course struc- 
ture. It is important that groups be large enough that there is a critical mass 
sufficient to take on learning issues, but groups that are either too large or too 
small impede the development of the necessary group dynamic. 

Some studies have suggested it is difficult for students to find a time to meet 
outside of the scheduled class. Our students were asked specifically not to discuss 
their research and learning issues outside of class but to wait until we were all 
present. This is essential for the integration of each individual’s research con- 
tribution to the whole group’s collective learning. Moreover, it is important for 
each student to have the experience of presenting her/his research to the group 
and to receive feedback in order to develop presentation skills. Students are also 
then afforded the opportunity to learn how to listen and to debate effectively 
and respectfully in the context of a group. When presenting and assessing the 
presentations of others is an iterative process, students have the opportunity to 
develop different critical and rhetorical strategies. Group feedback helps students 
to learn to recognize how to present core information and key points and relate 
these to the problem at hand. Working in the formal group means that the stu- 
dents benefit from group processing, which includes critiques of their presenta- 
tion skills and the quality of their research. Although there does not appear to be 
published data to support this, our experience indicates that containing discus- 
sions within the group meeting, followed by group processing, creates a higher 
functioning group than otherwise. Nevertheless, flexible scheduling is necessary 
to ensure that every member of the group (including facilitators) is present at 
eveiy class meeting. We were prepared to reschedule classes to accommodate 
travel, family emergencies, or illness. Students became so committed to the prin- 
ciple that every member be present that they were willing to meet at 7 :00 am or 
in the evening, if that was the only way to accommodate everyone’s schedule. 

A third factor key to the success of problem-based learning is the role of 
the facilitator(s). It is critical that facilitators do not convey content, provide 
necessary background, or lapse into mini-lectures. If facilitators assume the 
role of expert, students will come to count on them to provide the context, 
the necessary data, or even “the answers” to the problem. Perhaps even more 
importantly, facilitators need to be mindful always to stimulate the discus- 
sion through asking open-ended questions (Rhem, 1998). Questions that have 
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a “right” answer or, indeed, questions that reveal the fact that the facilitator is 
a content expert will elicit silence. These types of questions may also lead to 
the expectation that the facilitator will become the teacher and the leader and 
will ultimately take over the learning process. This will have a directly negative 
impact on the students’ experience and their learning. Students will then not 
only resist taking responsibility for the learning process, but they will begin 
to rebel against it and resent the facilitator’s expectation that they do research 
independently when ultimately they know they will be given “the answers.” 

Problem-based learning requires a considerable investment of time and 
imagination from the facilitators as they develop the problems prior to the start 
of the class. The focus of problem-based learning is not to solve the problem 
and find the correct answer. The best problems are ones that cannot be solved, 
but rather need to be understood from multiple perspectives offered by different 
disciplines, approaches, and experiences. This is central to one of the funda- 
mental goals of problem-based learning and, indeed, of a university education 
in general: the appreciation of the diversity of truths and the multiplicity of 
perspectives in all areas of knowledge. The complexity of problems and the 
web of behavioural learning issues that they generate should not lead to the 
conclusion that there can be no purpose or defined learning outcomes for a 
problem-based learning seminar. Indeed, the facilitators worked assiduously 
to ensure the learning outcomes of our course were met. These were not stated 
goals, however, and the students in our course never saw our rather ambitious 
list of learning outcomes (see Appendix 1). 

Group processing is perhaps the most important factor to the success of 
problem-based learning. It is central to the pedagogy and yet it is the aspect 
most frequently dropped from hybrid and partial applications of problem-based 
learning. Group processing functions to build trust and a sense of community 
and helps students develop the skills to work as members of a team. It is the 
mechanism by which students learn to evaluate realistically their strengths and 
weaknesses. Members of the group identily when someone exceeds expectations 
or cuts corners. However, it is vital that the facilitators model and encourage 
feedback that is focused and accurate. It is not adequate for members of the group 
simply to heap praise on one another. They have to appreciate their own skills 
and weaknesses, and use the feedback from others to find ways to improve their 
performance and participation. With consistent practice at group processing, stu- 
dents develop a sense of intellectual responsibility to themselves and each other, 
and come to appreciate the reciprocal responsibility they have for their learning. 
Facilitators must ensure that there is sufficient time available for group process- 
ing at the end of each meeting, but otherwise they should function as regular 
members of the group. Facilitators should model how to give and receive honest 
and constructive feedback. A high point in our experience was the meeting when, 
rather than being told we did not participate sufficiently, a member suggested 
that we were too involved in the discussion of learning issues. This marked a 
point when the students had truly taken ownership of their own learning. 
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CONCLUSION 

Although universities have adopted enthusiastically the language of learn- 
er-centred education and the goal of developing life-long independent learn- 
ers, there are few empirical studies that demonstrate success in meeting these 
objectives. This study demonstrates that students in a problem-based first-year 
seminar report significantly increased levels of skill development and enhanced 
educational experiences compared to their peers in small seminars or in con- 
ventional lecture courses. Moreover, the skills and abilities developed in the 
problem-based seminar proved to be transferable to other learning environ- 
ments and persisted throughout subsequent years. 

Problem-based learning addresses explicitly a number of the learning ob- 
jectives and the educational goals of Canadian universities. This pedagogical 
method enhances student experience and empowers students to take control 
of their learning - the goals inherent in the language of learner-centredness. 
Although benchmarking and Key Performance Indicators may be anathema to 
how most university faculty evaluate the goals and objectives of postsecondary 
education, there are few who would dissent from the expectations set out for 
undergraduate degrees. The problem-based learning format addresses all the 
indicators established for assessing university education, whether it be through 
students’ perception of their education, or through the application of objective 
qualitative criteria. 

This research also demonstrates that small class size does not in and of itself 
significantly enhance student experience or the acquisition of knowledge and 
the development of critical and analytical skills. It is only through a profound 
reshaping of our learning paradigms that there will be a fundamental change 
in learning and teaching. These research results demonstrate that there is as yet 
unrealized potential to enhance student education and experience through the 
implementation of problem-based learning techniques. Moreover, if one course 
has the ability to transform all subsequent learning, imagine how much more 
could be done with widespread application of problem-based learning across 
the curriculum at all levels. * 
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